A Stochastic Composite Augmented Lagrangian Method for Reinforcement Learning

In this paper, we consider the linear programming (LP) formulation for deep reinforcement learning. The number of constraints depends on size state and action spaces, which makes problem intractable in large or continuous environments. general augmented Lagrangian method suffers double-sampling obstacle solving program. Motivated from updates multipliers, overcome obstacles minimizing function by replacing conditional expectations with multipliers. Therefore, a parameterized is proposed. replacement provides promising breakthrough to integrate two steps into single quadratic penalty problem. A theoretical analysis shows that solutions generated sequence constrained optimization converge optimal solution program if error controlled properly. algorithm without using target networks under neural tangent kernel setting residual can be arbitrarily small parameter network chosen suitably. Preliminary experiments illustrate our competitive other state-of-the-art algorithms.